Distributed file systems offer many compelling advantages in establishing high performance computing environments. One example is the ability to easily expand, even at large scale. Another example is the ability to store different types of data, accessible by different types of clients, using different protocols. In servicing different sets of clients, a distributed file system may offer data services such as compression, encryption, off-site tiering, etc.
In many file systems, each file is associated with a single data stream. For example, a unique inode of the file can store metadata related to the file and block locations within specific storage disks where the file data is stored. When a client or other file system process desire access to a file, the unique inode associated with the file can be determined, and then the inode can be read as part of the processing the file system operation.
When a file system operation targeted to an inode is being processed, the inode itself can be placed under lock conditions, impacting other file system processes that desire access to the same inode. In addition, the size of an inode can be limited, such that when metadata relating to the file the inode is associated with grows too large, it may need to be stored elsewhere. For example, if an inode is associated with a file that has been tiered to an external storage repository, metadata may be generated that describes the location with the external storage repository for different chunks of file data, account information needed to access the external repository, etc.
One method to managing metadata associated with an inode is to store such metadata in user space, flag the inode within the kernel space that user space metadata exists that provides context to the inode, and then whenever a file system process needs to access the inode, the process can consult the user space metadata store for any necessary information to perform the file system operation. However, by keeping so much metadata and logic in user space, it can create inefficiencies and additional process expenses to maintain a separate metadata store in user space that provides further context to kernel space inodes.